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Software  for  Advanced  Vision  Systems 

Progress  Report  -  6/15/95 
Thomas  J.  Olson  and  Worthy  N.  Martin 


The  goal  of  the  ARPA  Advanced  Vision  Systems  program  (AVIS)  is  to  develop  hardware  ac¬ 
celerators  that  will  provide  dramatic  improvements  in  performance  for  machine  vision,  image 
processing,  automatic  target  recognition,  and  other  applications  with  similar  processing  needs. 
During  phase  I  of  the  program  our  group  conducted  a  study  of  software  issues  raised  by  the  AVIS 
program.  The  goal  of  our  current  effort  is  to  develop  an  abstract  lower  level  interface  for  real-time 
computer  vision  accelerators.  This  interface  will  allow  applications  or  development  environments 
to  control,  sequence  and  synchronize  with  computations  mnning  on  multiple  accelerators  without 
reference  to  vendor-  or  device-specific  functionality. 

Activities  to  date: 

We  delivered  the  final  Phase  I  report  on  April  15th.  One  of  the  lessons  of  that  effort  was  the 
importance  of  supporting  the  Khoros  image  processing  and  visualization  environment.  During  the 
past  three  months  we  have  been  working  to  understand  the  Khoros  process  model  and  identify  the 
options  for  supporting  it  on  non-standard  hardware.  We  plan  to  use  this  information  to  constrain 
the  design  of  the  abstract  low-level  API. 

Khoros  2.0  is  a  complete  rewrite  of  the  Khoros  system.  It  is  written  in  C,  but  uses  function  ta¬ 
bles  to  implement  essential  object-oriented  language  features  such  as  dynamic  inheritance.  Any 
approach  to  supporting  Khoros  on  AVIS  architectures  will  likely  make  use  of  these  features  for 
the  sake  of  maintainability  and  compatibility  with  other  Khoros  code.  There  appear  to  be  two  ba¬ 
sic  strategies  for  accessing  special  hardware  via  Khoros.  One  is  to  create  a  subclass  of  the  Khoros 
node  type.  Nodes  in  Khoros  encapsulate  the  computational  behavior  of  a  dataflow  graph  operator. 
It  would  not  be  difficult  to  create  a  node  that  spawns  child  processes  on  an  accelerator  to  do  the 
bulk  of  its  computation.  The  problem  with  this  approach  is  that  it  would  leave  gross  scheduling 
decisions  under  the  control  of  the  Khoros  executive. 

A  second  approach  to  using  special  hardware  via  Khoros  is  to  override  the  behavior  of  the 
Khoros  Workspace  object.  It  should  be  possible  to  create  workspaces  that  support  unique  sets  of 
nodes  and  schedule  them  according  to  unique  protocols  (e.g.  static  dependency  graph  scheduling). 
The  advantage  of  this  approach  is  that  it  should  make  it  possible  to  use  Khoros  operators  imple¬ 
mented  as  conventional  workstation  executables  in  combination  with  those  that  target  an  AVIS 
accelerator.  This  in  turn  would  provide  a  smooth  path  for  transitioning  existing  applications  to  the 
accelerator:  Khoros  operators  could  be  recoded  for  the  accelerator  one  at  a  time,  providing  incre¬ 
mental  performance  improvements. 

The  main  technical  problem  that  we  see  concerns  the  difficulty  of  using  multiple  accelerator- 
based  workspaces  in  a  single  application.  The  low-  and  intermediate-level  software  for  some 
AVIS  accelerators  may  require  a  global  view  of  the  computation,  e.g.  in  order  to  perform  static 
scheduling  and  resource  allocation.  Khoros  is  currently  conceived  as  a  highly  dynamic  system  in 
which  decisions  of  that  type  are  made  at  run  time.  We  are  currently  designing  a  Khoros  interface 
to  our  statically  scheduled  VEIL  system,  in  order  to  ensure  that  the  approach  we  choose  can  meet 
these  requirements. 
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